Deep-Learning-Based Whole-Lung and Lung-Lesion Quantification Despite Inconsistent Ground Truth: Application to Computerized Tomography in SARS-CoV-2 Nonhuman Primate Models

Rationale and Objectives Animal modeling of infectious diseases such as coronavirus disease 2019 (COVID-19) is important for exploration of natural history, understanding of pathogenesis, and evaluation of countermeasures. Preclinical studies enable rigorous control of experimental conditions as well as pre-exposure baseline and longitudinal measurements, including medical imaging, that are often unavailable in the clinical research setting. Computerized tomography (CT) imaging provides important diagnostic, prognostic, and disease characterization to clinicians and clinical researchers. In that context, automated deep-learning systems for the analysis of CT imaging have been broadly proposed, but their practical utility has been limited. Manual outlining of the ground truth (i.e., lung-lesions) requires accurate distinctions between abnormal and normal tissues that often have vague boundaries and is subject to reader heterogeneity in interpretation. Indeed, this subjectivity is demonstrated as wide inconsistency in manual outlines among experts and from the same expert [1]. The application of deep-learning data-science tools has been less well-evaluated in the preclinical setting, including in nonhuman primate (NHP) models of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection/COVID-19, in which the translation of human-derived deep-learning tools is challenging. The automated segmentation of the whole lung and lung lesions provides a potentially standardized and automated method to detect and quantify disease. Materials and Methods We used deep-learning-based quantification of the whole lung and lung lesions on CT scans of NHPs exposed to SARS-CoV-2. We proposed a novel multi-model ensemble technique to address the inconsistency in the ground truths for deep-learning-based automated segmentation of the whole lung and lung lesions. Multiple models were obtained by training the convolutional neural network (CNN) on different subsets of the training data instead of having a single model using the entire training dataset. Moreover, we employed a feature pyramid network (FPN), a CNN that provides predictions at different resolution levels, enabling the network to predict objects with wide size variations. Results We achieved an average of 99.4 and 60.2% Dice coefficients for whole-lung and lung-lesion segmentation, respectively. The proposed multi-model FPN outperformed well-accepted methods U-Net (50.5%), V-Net (54.5%), and Inception (53.4%) for the challenging lesion-segmentation task. We show the application of segmentation outputs for longitudinal quantification of lung disease in SARS-CoV-2-exposed and mock-exposed NHPs. Conclusion Deep-learning methods should be optimally characterized for and targeted specifically to preclinical research needs in terms of impact, automation, and dynamic quantification independently from purely clinical applications.

Deep-Learning-Based Whole-Lung and Lung-Lesion Quantification Despite Inconsistent Ground Truth: Application to Computerized Tomography in SARS-CoV-2 Nonhuman Primate Models.

RATIONALE AND OBJECTIVES: Animal modeling of infectious diseases such as coronavirus disease 2019 (COVID-19) is important for exploration of natural history, understanding of pathogenesis, and evaluation of countermeasures. Preclinical studies enable rigorous control of experimental conditions as well as pre-exposure baseline and longitudinal measurements, including medical imaging, that are often unavailable in the clinical research setting. Computerized tomography (CT) imaging provides important diagnostic, prognostic, and disease characterization to clinicians and clinical researchers. In that context, automated deep-learning systems for the analysis of CT imaging have been broadly proposed, but their practical utility has been limited. Manual outlining of the ground truth (i.e., lung-lesions) requires accurate distinctions between abnormal and normal tissues that often have vague boundaries and is subject to reader heterogeneity in interpretation. Indeed, this subjectivity is demonstrated as wide inconsistency in manual outlines among experts and from the same expert. The application of deep-learning data-science tools has been less well-evaluated in the preclinical setting, including in nonhuman primate (NHP) models of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection/COVID-19, in which the translation of human-derived deep-learning tools is challenging. The automated segmentation of the whole lung and lung lesions provides a potentially standardized and automated method to detect and quantify disease. MATERIALS AND METHODS: We used deep-learning-based quantification of the whole lung and lung lesions on CT scans of NHPs exposed to SARS-CoV-2. We proposed a novel multi-model ensemble technique to address the inconsistency in the ground truths for deep-learning-based automated segmentation of the whole lung and lung lesions. Multiple models were obtained by training the convolutional neural network (CNN) on different subsets of the training data instead of having a single model using the entire training dataset. Moreover, we employed a feature pyramid network (FPN), a CNN that provides predictions at different resolution levels, enabling the network to predict objects with wide size variations. RESULTS: We achieved an average of 99.4 and 60.2% Dice coefficients for whole-lung and lung-lesion segmentation, respectively. The proposed multi-model FPN outperformed well-accepted methods U-Net (50.5%), V-Net (54.5%), and Inception (53.4%) for the challenging lesion-segmentation task. We show the application of segmentation outputs for longitudinal quantification of lung disease in SARS-CoV-2-exposed and mock-exposed NHPs. CONCLUSION: Deep-learning methods should be optimally characterized for and targeted specifically to preclinical research needs in terms of impact, automation, and dynamic quantification independently from purely clinical applications.

Toward the determination of sensitive and reliable whole-lung computed tomography features for robust standard radiomics and delta-radiomics analysis in a nonhuman primate model of coronavirus disease 2019.

Purpose: We propose a method to identify sensitive and reliable whole-lung radiomic features from computed tomography (CT) images in a nonhuman primate model of coronavirus disease 2019 (COVID-19). Criteria used for feature selection in this method may improve the performance and robustness of predictive models. Approach: Fourteen crab-eating macaques were assigned to two experimental groups and exposed to either severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) or a mock inoculum. High-resolution CT scans were acquired before exposure and on several post-exposure days. Lung volumes were segmented using a deep-learning methodology, and radiomic features were extracted from the original image. The reliability of each feature was assessed by the intraclass correlation coefficient (ICC) using the mock-exposed group data. The sensitivity of each feature was assessed using the virus-exposed group data by defining a factor R that estimates the excess of variation above the maximum normal variation computed in the mock-exposed group. R and ICC were used to rank features and identify non-sensitive and unstable features. Results: Out of 111 radiomic features, 43% had excellent reliability ( ICC > 0.90 ), and 55% had either good ( ICC > 0.75 ) or moderate ( ICC > 0.50 ) reliability. Nineteen features were not sensitive to the radiological manifestations of SARS-CoV-2 exposure. The sensitivity of features showed patterns that suggested a correlation with the radiological manifestations. Conclusions: Features were quantified and ranked based on their sensitivity and reliability. Features to be excluded to create more robust models were identified. Applicability to similar viral pneumonia studies is also possible.

Toward the determination of sensitive and reliable whole-lung computed tomography features for robust standard radiomics and delta-radiomics analysis in a nonhuman primate model of coronavirus disease 2019

. Purpose We propose a method to identify sensitive and reliable whole-lung radiomic features from computed tomography (CT) images in a nonhuman primate model of coronavirus disease 2019 (COVID-19). Criteria used for feature selection in this method may improve the performance and robustness of predictive models. Approach Fourteen crab-eating macaques were assigned to two experimental groups and exposed to either severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) or a mock inoculum. High-resolution CT scans were acquired before exposure and on several post-exposure days. Lung volumes were segmented using a deep-learning methodology, and radiomic features were extracted from the original image. The reliability of each feature was assessed by the intraclass correlation coefficient (ICC) using the mock-exposed group data. The sensitivity of each feature was assessed using the virus-exposed group data by defining a factor R that estimates the excess of variation above the maximum normal variation computed in the mock-exposed group. R and ICC were used to rank features and identify non-sensitive and unstable features. Results Out of 111 radiomic features, 43% had excellent reliability ( Conclusions Features were quantified and ranked based on their sensitivity and reliability. Features to be excluded to create more robust models were identified. Applicability to similar viral pneumonia studies is also possible.

The Evolution of Medical Countermeasures for Ebola Virus Disease: Lessons Learned and Next Steps.

The Ebola virus disease outbreak that occurred in Western Africa from 2013-2016, and subsequent smaller but increasingly frequent outbreaks of Ebola virus disease in recent years, spurred an unprecedented effort to develop and deploy effective vaccines, therapeutics, and diagnostics. This effort led to the U.S. regulatory approval of a diagnostic test, two vaccines, and two therapeutics for Ebola virus disease indications. Moreover, the establishment of fieldable diagnostic tests improved the speed with which patients can be diagnosed and public health resources mobilized. The United States government has played and continues to play a key role in funding and coordinating these medical countermeasure efforts. Here, we describe the coordinated U.S. government response to develop medical countermeasures for Ebola virus disease and we identify lessons learned that may improve future efforts to develop and deploy effective countermeasures against other filoviruses, such as Sudan virus and Marburg virus.

Clinical grade ACE2 as a universal agent to block SARS-CoV-2 variants.

The recent emergence of multiple SARS-CoV-2 variants has caused considerable concern due to both reduced vaccine efficacy and escape from neutralizing antibody therapeutics. It is, therefore, paramount to develop therapeutic strategies that inhibit all known and future SARS-CoV-2 variants. Here, we report that all SARS-CoV-2 variants analyzed, including variants of concern (VOC) Alpha, Beta, Gamma, Delta, and Omicron, exhibit enhanced binding affinity to clinical grade and phase 2 tested recombinant human soluble ACE2 (APN01). Importantly, soluble ACE2 neutralized infection of VeroE6 cells and human lung epithelial cells by all current VOC strains with markedly enhanced potency when compared to reference SARS-CoV-2 isolates. Effective inhibition of infections with SARS-CoV-2 variants was validated and confirmed in two independent laboratories. These data show that SARS-CoV-2 variants that have emerged around the world, including current VOC and several variants of interest, can be inhibited by soluble ACE2, providing proof of principle of a pan-SARS-CoV-2 therapeutic.

Duplex One-Step RT-qPCR Assays for Simultaneous Detection of Genomic and Subgenomic RNAs of SARS-CoV-2 Variants.

A hallmark of severe acute respiratory syndrome virus (SARS-CoV-2) replication is the discontinuous transcription of open reading frames (ORFs) encoding structural virus proteins. Real-time reverse transcription PCR (RT-qPCR) assays in previous publications used either single or multiplex assays for SARS-CoV-2 genomic RNA detection and a singleplex approach for subgenomic RNA detection. Although multiplex approaches often target multiple genomic RNA segments, an assay that concurrently detects genomic and subgenomic targets has been lacking. To bridge this gap, we developed two duplex one-step RT-qPCR assays that detect SARS-CoV-2 genomic ORF1a and either subgenomic spike or subgenomic ORF3a RNAs. All primers and probes for our assays were designed to bind to variants of SARS-CoV-2. In this study, our assays successfully detected SARS-CoV-2 Washington strain and delta variant isolates at various time points during the course of live virus infection in vitro. The ability to quantify subgenomic SARS-CoV-2 RNA is important, as it may indicate the presence of active replication, particularly in samples collected longitudinally. Furthermore, specific detection of genomic and subgenomic RNAs simultaneously in a single reaction increases assay efficiency, potentially leading to expedited lucidity about viral replication and pathogenesis of any variant of SARS-CoV-2.

Medical imaging of pulmonary disease in SARS-CoV-2-exposed non-human primates.

Chest X-ray (CXR), computed tomography (CT), and positron emission tomography-computed tomography (PET-CT) are noninvasive imaging techniques widely used in human and veterinary pulmonary research and medicine. These techniques have recently been applied in studies of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-exposed non-human primates (NHPs) to complement virological assessments with meaningful translational readouts of lung disease. Our review of the literature indicates that medical imaging of SARS-CoV-2-exposed NHPs enables high-resolution qualitative and quantitative characterization of disease otherwise clinically invisible and potentially provides user-independent and unbiased evaluation of medical countermeasures (MCMs). However, we also found high variability in image acquisition and analysis protocols among studies. These findings uncover an urgent need to improve standardization and ensure direct comparability across studies.

Animal models for COVID-19.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the aetiological agent of coronavirus disease 2019 (COVID-19), an emerging respiratory infection caused by the introduction of a novel coronavirus into humans late in 2019 (first detected in Hubei province, China). As of 18 September 2020, SARS-CoV-2 has spread to 215 countries, has infected more than 30 million people and has caused more than 950,000 deaths. As humans do not have pre-existing immunity to SARS-CoV-2, there is an urgent need to develop therapeutic agents and vaccines to mitigate the current pandemic and to prevent the re-emergence of COVID-19. In February 2020, the World Health Organization (WHO) assembled an international panel to develop animal models for COVID-19 to accelerate the testing of vaccines and therapeutic agents. Here we summarize the findings to date and provides relevant information for preclinical testing of vaccine candidates and therapeutic agents for COVID-19.

Decreases in cardiac catheter laboratory workload during the COVID-19 level 4 lockdown in New Zealand.

An increase in coronavirus disease (COVID-19) infections prompted Level 4 lockdown throughout New Zealand from 25 March 2020. We have investigated trends in coronary and electrophysiology (EP) procedures before and during this lockdown. The number of acute procedures for ST elevation myocardial infarction remained stable. In contrast, the number of in-patient angiograms and percutaneous intervention procedures fell by 53% compared with the previous 4 weeks in 2020 and by 56% compared with the corresponding period in 2019. Further study is required to determine the reasons for these trends.